Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Typst Language Support #302

Open
wants to merge 49 commits into
base: master
Choose a base branch
from
Open

feat: Typst Language Support #302

wants to merge 49 commits into from

Conversation

grantlemons
Copy link
Contributor

@grantlemons grantlemons commented Dec 4, 2024 •
edited
Loading

Re-opening of #289 from my fork instead of a branch

This was referenced Dec 4, 2024
Closed
Closed
Merged
@grantlemons
feat(Automattic#230): map basic typst expressions to tokens
bed51f7
@grantlemons
feat(Automattic#230): change recursive shorthand from macro to function
7201575
@grantlemons
feat(Automattic#230): flesh out more complicated typst syntax parsing
d880613
@grantlemons
feat(Automattic#230): delegate typst files to parser in harper-cli an…
9376e71 
…d harper-ls
@grantlemons
fix(Automattic#230): fix offset update after delegating parser
749e6dd
@grantlemons
fix(Automattic#230): ParBreak to ParBreak, not two Newlines
ba3c307
@grantlemons
feat(Automattic#230): remove offset variable, and just use the start …
c6a4d05 
…of an

environment's span
@grantlemons
feat(Automattic#230): parse numbers properly and add test for numbers
0425110
@grantlemons
feat(Automattic#230): consolidate words separated by apostrophes into…
1f43b27 
… possessives or conjunctions
@grantlemons
fix(clippy): satisfy clippy
835c396
@grantlemons
feat(Automattic#230): simplify possessive-conjunction logic and add r…
4dbc264 
…espective tests
@grantlemons
feat(Automattic#230): create additional parsers for complex dictionar…
f3eda92 
…y parsing
@grantlemons
feat(Automattic#230): add some tests for dictionary parsing, and impr…
24e0551 
…ove dict parsing to fit better
@grantlemons
fix(Automattic#230): fix dict parsing by manually getting document co…
c63d41a 
…ntent in span so quotes aren't escaped
@grantlemons
fix(Automattic#230): remove debug print of typst ast in test
f57d6c2
@grantlemons
style(Automattic#230): expand explainer on str parsing
550cf20
@grantlemons
Copy link
Contributor Author

grantlemons commented Dec 18, 2024 •
edited
Loading

Mostly just needs more tests. I'm no expert in Typst syntax, so if someone wants to help it would be much appreciated.

@Andrew15-5
Copy link

If I can understand the core concepts mentioned above (what should/could be checked), then I maybe be able to add some tests.

@grantlemons
fix: better handling of spaces and newlines
b61c78c 
- Only consolidate adjacent space spans
- Typst spaces distinguish between newlines and spaces
- Typst spaces count number of spaces
- Spans command now defaults to not show newlines
@grantlemons
fix: testing rig takes a parser instead of assuming markdown
a29b879 
- Remove unneeded &mut impl Parser in document
@grantlemons
fix(Automattic#230): mark numbers outside text as unlintable
00623b0
@grantlemons
tests(Automattic#230): add document with spelling mistakes in differe…
122436f 
…nt contexts
@grantlemons grantlemons marked this pull request as ready for review December 30, 2024 00:49
@grantlemons grantlemons mentioned this pull request Dec 30, 2024
Draft
elijah-potter
elijah-potter requested changes Dec 30, 2024
View reviewed changes
Copy link
Collaborator

@elijah-potter elijah-potter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but I have some questions.

harper-cli/src/main.rs Outdated Show resolved Hide resolved
harper-tree-sitter/src/lib.rs Show resolved Hide resolved
harper-core/Cargo.toml Outdated
@@ -24,6 +24,7 @@ thiserror = "2.0.9"
unicode-blocks = "0.1.9"
unicode-width = "0.2.0"
levenshtein_automata = { version = "0.2.1", features = ["fst_automaton"] }
typst-syntax = "0.12.0"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typst is pretty big and completely unused for certain applications (i.e. Obsidian). Would you mind putting Typst support behind a feature flag?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, you could make the Typst parser its own crate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would making it a seperate crate help? It would still be a dependency...

I feel like feature flag is the way to go. I assume we would have it enabled for harper-ls and harper-cli?

Copy link
Collaborator

@elijah-potter elijah-potter Jan 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple reasons.

The intention is eventually, and especially with harper-tree-sitter, that Parsers may become a community endeavor. They should be able to be built without great understanding of Harper internals. Such parsers should be able to stand on their own without official support from me or Automattic.

If people build applications (other than harper-ls) using Harper, they should be able to build and pull in third-party parsers as they need. Having many parsers in harper-core sets the expectation that all parsers must be officially supported and maintained by us.

Eventually, I hope this to be true for Linters as well, but that's a little ways off.

In fact, if you wish to move the Typst Parser to its own repository and make this PR simply import it, I find that most appealing.

harper-core/src/mask/mod.rs Show resolved Hide resolved
@grantlemons
Revert "feat(spans-visual): mark unlintable spans as red"
2e97afe 
This reverts commit 3f9ecb5.
@grantlemons
Revert portions of "fix: better handling of spaces and newlines"
83789eb 
This reverts part of commit b61c78c.
The portion reverted is the default hiding of newlines in the span
command, which has been relocated to a new branch
@grantlemons grantlemons mentioned this pull request Dec 30, 2024
Merged
@grantlemons grantlemons mentioned this pull request Dec 31, 2024
Open
@elijah-potter elijah-potter mentioned this pull request Jan 3, 2025
Open
@grantlemons
Copy link
Contributor Author

I decided to move typst parsing into its own crate, since it simplifies the feature dependency and establishes good precident that can be followed for unofficial/external parsers in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattfbacon mattfbacon mattfbacon left review comments

@Andrew15-5 Andrew15-5 Andrew15-5 left review comments

@elijah-potter elijah-potter Awaiting requested review from elijah-potter

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@grantlemons @Andrew15-5 @elijah-potter @mattfbacon